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NONUNIFORM BEHAVIOR AND STABILITY OF HOPFIELD 
NEURAL NETWORKS WITH DELAY 

ANTONIO J. G. BENTO, JOSE J. OLIVEIRA, AND CESAR M. SILVA 


Abstract. We obtain a result on the behavior of the solutions of a general 
nonautonomous Hopfield neural network model with delay, assuming some 
general bound for the product of consecutive terms in the sequence of neu¬ 
ron charging times and some conditions to control the nonlinear part of the 
equations. We then apply this result to improve some existent results on the 
stability of Hopfield models. Our results are based on a new abstract result 
on the behavior of nonautonomous delayed equations. 


1. Introduction 

Due to their many applications in various engineering and scientific areas such 
as signal processing, image processing and pattern classification (see [Sill]), neural 
network models are still a subject of active research. Concerning neural networks, 
one of the most important objects of study is the obtention of conditions guaran¬ 
tying the global stability of equilibrium states [niiiKis], periodic solutions din] 
or, more generally, of a particular solution [8]. 

In the present work we consider a discrete-time nonautonomous neural network 
with time delay. The relevance of our setting is easily clarified. Firstly, although 
theoretically speaking neural networks should be described by a continuous-time 
model, it is essential to formulate discrete-time versions that can be implemented 
computationally mm]. Secondly, it is important to consider delay in modelling 
neural networks in order to reproduce the effect of finite transmission speed of 
signals among neurons (there is a mathematical counterpart of this since time delay 
may cause instability and oscillation [10]). At last, nonautonomy is associated to 
the change of parameters such as neuron charging time, interconnection weights 
and external inputs in the course of time. This nonautonomy can be translated 
not only by time-varying parameters, but also by time-varying delays [HllISlIIl]- 
There are still few stability results in the context of nonautonomous nonperiodic 
neural network models m- 

Unlike the usual assumptions, our conditions correspond to bound the product 
of consecutive terms in the sequence of neuron charging times and to assume some 
condition controlling the nonlinear part of the equations. The classic method of 
proof is to consider that there is an equilibrium point or a periodic solution and 
to construct a suitable Lyapunov function to assure the global stability of this 
particular solution [a ig HU Us] US). The technique used here is different from 
the usual ones, namely we see our system as a sufficiently small perturbation of 
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a nonuniform contraction and use Banach’s fixed point theorem in some suitable 
complete metric space to obtain the global stability of our system. This approach 
allows us to dismiss the requirement of existence of a stationary or, more generally, 
periodic solution and additionally to consider a more general form for the nonlinear 
part of the model. When we restrict to the particular case of a periodic Hopfield 
model, our conditions for existence of a globally stable periodic solution generalize 
the existent results in the literature m- 

This work is organized in the following way. In section[2]we use the discretization 
technique in m to obtain a discrete version of a generalized neural network model 
that includes the well known Hopfield neural network models considered in [HUT] 
and the bidirectional associative memory neural network models studied in HIE]. 
Next, we state our main stability result and we see that it is a consequence of the 
abstract result considered in section [3l As a corollary, we get global exponential 
stability for the models in m under distinct hypothesis from the ones assumed in 
that paper. After, for the periodic model, considering a Poincare map, we obtain the 
existence of a periodic solution as a consequence of the global exponential stability. 
This result improves one of the main results in m- Finally, in section|3l we consider 
general discrete-time delayed models that include our neural network models as 
particular cases and obtain an abstract global stability result that establishes in 
particular the stability results in section O 


2. Hopfield Models 


As a generalization of the continuous-time Hopfield neural network models pre¬ 
sented in [Ml Hz] we have 

N 

x',{t) =-ai{t)xi{t)+ - aij{t))), t ^ 0, i = 1,... ,N, (1) 

j=i 

where Oi : [0,-|-oo[—)• [0,-|-oo[, kij : [0,-|-oo[x]R —>■ R, and : [0,-|-oo[^- [0,-|-oo[ 
are continuous functions with aij bounded and kij Lipschitz on the second variable. 
Here ai{t) is the neuron charging time. 

Following the ideas in m, to obtain a discrete-time analogue of the continuous¬ 
time model o, we consider the following approximation 


N 


x'iit) = —ai{\t/h]h)xi(t) + '^2,^i3 ( [t/h]h,Xj i [t/h]h — 


1=1 


h 



i = 1,..., N, t G [mh, {m + l)/i[ for m G No, where ft, is a fixed positive real number 
(discretization step size) and [r] denotes the integer part of the real number r. 
Clearly, for t G [mft, (to -|- l)ft[ we have [t/ft] = to and the model ([2]) has the form 

x'i{t) = —ai(mh)xi{t) + ^ kij ^to/i, Xj ^ ^to — 



which is equivalent to 


N 

-I- ai(TOft)e“‘^™^^‘xi(t) = ^ kij ( mh,Xj ((to — Tij { m )) ft)) 

j=i 


( 3 ) 








NONUNIFORM BEHAVIOR AND STABILITY OF HOPFIELD... 


3 


where 


,(m) = 


j [mh) 


Integrating over [mh^t[^ with t < (m + l)h, we obtain 

pt 


^ai {mh)s 


Xi{s) 


ds = 


J mh ^ 
which is equivalent to 


^ai{mh)t _ ^ai{mh)mh'^ 


i(mh) 


^ _ ^ai{mh){mh—t) 


{mh,Xj ((m — Tij{m)) h)). 
i=i 


-——-j {mh,xj {{m - n,(m)) h)). 


N 


Letting t —>■ (m + l)/i, we obtain 




2 _ ^—ai{mh)h 

ai(mh) 


N 


kij {mh, Xj ((to — Tij{m)) h)). 
i=i 

( 4 ) 

Thus, identifying Xi{mh) with Xi{m), ai{mh) with ai{m) and kij{mh, •) with 
kij{m, ■) and defining 

1 _ p-ai(m)h 

0.{m) = -(5) 

ai{m) 


equation (j4|) becomes 

N 

Xi(TO + 1) = e““’^™^^Xi(TO) + 9i{m) fcy [m,Xj{m — Tij{m))). (6) 

i=i 

The model ® can be rewritten in the following way 

N 

Xi{m + 1) = Ci{m)xi{m) + hij(m,Xj{m - Tij{m))), (7) 

i=i 

i = 1,..., N, m € No, where ct : No ^]0,1[, : Nq Nq are bounded functions 

with r := max{rij(TO) : to G N, *, j = 1,..., TV}, and hy : No x K ^ R are Lipschitz 
functions on the second variable, i.e., there exist Hij : Nq —>■ R.'*' such that 

jhij(m, u) — hij{m, u)| ^ Hij{m)\u — u|, \/u,v G R, to G No- 

In this paper we consider the Hopfield neural network model ® that generalizes 
some existent models in the literature [3 El m HU. 

Before stating our main result, we need to introduce some notation. Let 

A = {(to, n) G : to ^ n ^ O} 

and, given a set / C R and a number r G Z“, define /z = / fl Z. Consider the space 
X of the functions 

a : [r, 0]z -)• R 

all = max |a(j)|. 

j=r,...,0 


equipped with the norm 
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Given iV S N, we are going to consider the cartesian products and equipped 
with the supremum norm, i.e., for a = (oi,..., oat) S and y = {yi ,..., j/at) € 
we have 

||a|| = max lloill = max ( max |ai(j)| 

1=1,...,N 1=1,...,N \j=r,...,0 


and 


\y\ = max 
i=l,...,N 

Given n € Nq and a function x: [n + r, +oo[z^ we denote the tth component 

by Xi, i.e., x = {xi,.. .xm)- For each m € No such that m ^ n, we define Xm S X^ 

by 

XmU) = s(m + j), j = r,r + 1,... ,0. 

For each n G Nq and each a G X^, we denote by x{-,n,a) the unique solution 

X : [n + r, +oo[z—>• R^ 


of ([7]) with initial conditions Xn = a. 

We now state our main global stability result for the neural network model given 
by 0. This theorem furnishes a bound for the distance between solutions of © 
based on bound for the products of consecutive neuron charging times, assuming 
that the Lipschitz constants of the nonlinear part of the model are sufficiently small. 
We will use it to obtain several results on the stability of several neural networks. 


Theorem 1. Consider model 0 and assume that there exist a double sequence 

Wm,n)(m,n)eA SUch that 


m—1 

s—n 


( 8 ) 


for all i = 1,..., N and all (m, n) G A, and 

m — 1 


X := max 


1 


N 


sup 


(m,n)eA I 

Then, for every a, a* : [r,0]z —S' R^ and every {m,n) G A, we have 

1 , 


< 1 . 


\\xm{,-,n,a) - Xm{-,n,a*)\\ ^ 




Proof. Gonsider n G Nq and a, a* : [r, 0]z —>• R'^. The change 

y{m) = x{m, n, a) — x(m, n,a*) 
transforms © into the system 


N 

yi{m + 1) = Ci{m)yi{m) + hij{m,yj{m - Tij{m))), (9) 

i=i 

i = 1,..., N, m n, where 

hijijn, u) = hij (m, u + Xj(rn — Tij{m),n, a*)) — hij{m, Xj{m — Tij{m),n,a*)). 

Now, y = 0 is an equilibrium point of (l9|) and, by Theorem [5l we obtain, for all 
function /3 : [r, 0]z —J- R^, 


\\ymi-,n,l3)\\ ^ 


1-A 
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for all m ^ n. Letting P = a — a*, we conclude that 

\\xm{-,n,a) -Xm{-,n,a*)\\ = \\y^{-,n,a-a*)\\ < H 

for all m ^ n. □ 


In |17] the authors considered the following discretization of a nonautonomous 
continuous-time Hopfield neural network model, which is a particular case of 


Xi{m + 1 ) 


x,{m) 


N 

b^j{m)fj{xj{m - t(to))) -k h{m) 
j=i 


, ( 10 ) 


i = 1,... ,7V, where ai,bij,Ii,T : No —>■ K are bounded functions with ai(m) > 0, 
0 ^ rirn) ^ r, /j : R —5> K. are Lipschitz functions with Lipschitz constant Fj > 0, 
9i (m) is given by ([5]) and h > 0 (/i is the discretization step size). We are going to 
use the following notation 

a“ = inf ai(TO) and bf, = sup \bij{m)\ and 6f =sup9i{m). 


We have the following result that establishes the global exponential stability of 
all solutions of m- 

Corollary 2. If 

N 

for every i = 1,... ,N, then model (na is globally exponentially stable, i.e., there 
are constants /i > 0 and C > 1 such that 

\\xm{-,n,a) -Xjn{-,n,a*)\\ < ||a-a*|| 

for every a, a* : [—t, 0]z —)• R'^ and every {m,n) G A. 

Proof. We will show that we are in the conditions of Theorem[TJ Defining Vi = a~h, 
by m there is positive number < min Ui such that 


Vi = l,...,N. (12) 

^ -1 

Putting a(„ „ = Q-n{m-n) ^ condition (jS]) is trivially satisfied because 

(i) ^ -i'i(m-n) ^ 

^m,n ^ ^ ^ 
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1 — 

Since ^ ;—, we have by m that 


A = max 

2=1,...,iV 


^ max 

2=1,...,iV 


< max 

2=1,...,iV 


= max 
2=1,...,AT 


= max 

2=1,...,iV 


sup < 
(m,n)C A 

\ 1 

[ k=n 

sup < 

(m,n) ^ A 

^ m—l 

^ k—n 

m — k — '. 

sup < 

(m,n)€ A 

'm-l 

Q{vi-h){k-Tn) 

. k—n , 


sup < 

(m,n)€ A 

1 _ - 

ie-^ 



sup < 

(m,n)€ A 

1 _ Q{’'i-n)in-m) 1 



N 

ik)Y,Kik)\i 

i=i 

l)-fj,{k-n) I 0 + ^ ^ 

J i=i 

1 - e-*^* -1 

e''* —1 


a, 


{e’^' -l)a. 


= 1 


and this proves the corollary. 


□ 


In the next corollary we slightly improve condition (HU in the last corollary. 
To do that we need to define the concept of an M-matrix. We say that a square 
real matrix is an M-matrix if the off-diagonal entries are nonpositive and all the 
eigenvalues have positive real part. 

Now consider the N x iV-matrix A4 defined by 

M = diag{a^, [bfjFj] 

Corollary 3. If Ai is an M-matrix, then the model (1101) is global exponential stable, 
i.e., there are g, > 0 and C > 1 such that 

\\xm{,-,n,a) -Xm{-,n,a*)\\ < ||a-a*||. 

for every a, a* : [—r, 0]z —> and every {m,n) G A. 

Proof. If A4 is an M-matrix, then (see 0 Theorem 5.1]) there is d = (di,..., djv) > 
0 such that M.d> 0, i.e., 

N 

(13) 

i=i 

The change yi(rn) = d~^Xi{m), m S No and i = 1,..., N, transforms (HQl) into 


yi{m1) = yi{m) e +6»i(m) 


N 

E 

i=i 


k^j{m)fj{yj{m - T{m))) -f h{m) 


bij{m) = d^ ^bij{m), fj{u) = fj{dju), and him) = d^ ^him), 


where 
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for TO G No and u G K. As fj are Lipschitz functions with constant Fj, then fj are 
also Lipshitz functions with constant Fj = djFj. From (IT^ we have 

N 

i=i 


which is equivalent to 


N 


0-i 

i=i 

and the result follows from the Corollary [5J 


□ 


Now we improve [HI Theorem 4.2], which proves the existence and global stabil¬ 
ity of the periodic solution of the Hopfield neural network model (IlOp with periodic 
coefficients. Let w G N and consider the model CHD where a^, 6^, 7^, r : Nq —>■ K are 
w-periodic functions. 

Theorem 4. If A4 is a M-matrix, then the model cni) has a unique oj- periodic 
solution which is globally exponentially stable. 

Proof. Let n G Nq. From Corollary [31 there are /r > 0 and C > 1 such that 

\\xm{-,n,a) -Xm{-,n,a*)\\ ^ C||a - a*||, (14) 

for all TO ^ n and all a, a* G . Now, choosing an integer fc G N such that 

< 1 (15) 

and defining the map P : —>• X^ by P{a) = Xn +tj(-, n, a). For a, a* G X^, we 

have 

||P'=(a)-P'=(a*)|| 

= \\P{P'^-\a)) - P{P’^-\a*))\\ 

= \\xn-i-u,{-,n,P'"~^{a)) -Xn-i-c.j{-,n,P'"~^{a*))\\ 

— Il^n-t-o; -i^i Xn+u) (^))) Xn+oj (‘j Xnj-u} (^ )))ll; 

and, as the model (nni is w-periodic, from m, 

||P'=(a) -P'=(a*)|| = \\x„+ 2 U;n,P'^-Ha)) - x„+ 2 ^(-, n, P'=-2(a*))|l 

— Il^n-t-fcw (* 5 Xnj-kcu (‘j ) II 

< Ce-'^'=‘" ||a-a1|. 

From C3, the map P^ is a contraction on X^. As X^ is a Banach space, we 
conclude that there is a unique point ^ G X^ such that P^ffp) = Noting that 

P'=(P(^))=P(P'=(^))=P(^), 

we have P(jp) = Tp which means Xn+uii'-, n,Tp) = Ip. 

Finally, as x{m,n,p) is a solution of (fTOll with ai,bij,Ii,T w-periodic functions, 
we know that x{m + w, n,^) is also a solution of (1101) and 

x{m,n,p) =x(ni,n,Xn+ui[-,n,p)) =x{'mFw,n,Tp) 

for every m ^ n. Therefore x(m, n, p) is a w-periodic solution of (1101) and, from 
da, all other solutions converge to it with exponential rates. □ 
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3. Stability of nonuniform contractions 


Let y be a Banach space and denote by X the space of functions 

OL : [r, 0]z -t Y 

equipped with the norm ||a|| = max |Q;(j)|, where | • | is the norm in Y. Given 

jeh,o]z 

N gN, we are going to consider the cartesian products and Y^ equipped with 
the supremum norm, i.e., for a = (ai ,... ,aN) G X^ and y = (j/i,...,?/ 7 v) S Y^, 
we have 


and 


|a|| = inax llaill = max max |ai(j)| 


\y\ = max 


Given a function x: [r, +oo[z^ Y^ we define, for each m G No, Xm G X^ by 
XmU) = + j), j = r, r + 1,..., 0. 

Let : X^ —)• Y^ be Lipschitz perturbations such that 

7mi0) = 0 (16) 


for every m G Nq. 

We are going to consider the following nonlinear delay difference equation 

xijn y 1) — LjYiXjYi -\~ f ^ ^ No, (11^) 

where, for each m G No, Lm ■ X^ —>• Y^ is given by 



with Lm : X —>• y a bounded linear operator for i = 1,... ,iV. For each n G No 
and a G X^, we obtain a unique function 

x: [n + r, +oo[z-)> y^, 

denoted by x(-,n,a), such that Xn — a and (ED holds. Gonsequently, for each 
(m, n) G A, we can define the operator Jm,™ : —>• X^ given by 

ILm,n (c) — U, c). Cl G X 

Associated with equation (II3, we will consider the linear difference equation 

Vi{m+\) = (19) 

i = where Vi^m G A is defined, as usual, by Vi^rn(j) = Vi(rn + j), j = 

r, r + 1,..., 0 and Lm is given by (IT^ . For each n G No and ai G X, we obtain a 
unique function Vi : [n + r, +oo[z—>■ y, denoted by Vi(-, n, ai), such that Vi^n = ca 
verifying m- 

For each (m, n) G A and i = 1,..., N, we define the operator : A —>• A by 

^i), (^i G A. 

We can easily verify that 

a) Am]n is linear for each (m, n) G A; 

b) = Id; 

c) for (I, m), (m, n) G A. 
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It is easy to prove by induction in m (see [T]) that 

where, for i = 1 ,..., 

m —1 

k—n 

a = (ai,..., Oat), fk^k) = (^fi^\xk), • ■ ■, and T: F ^ X is defined by 


Tu: [r,0]z ^ F 

j HA ru(j) = 


u if j = 0, 
0 ifj<0, 


for all u GY. 

Theorem 5. Let —>■ be Lipschitz functions such that dUD is satisfied 


and consider equation (flTll . Let (^am,n^ 
be double sequences such that 


m,n)G A 


,i = l,...,N, and let (a(„„) 


(m,n) ^ A 


||/](*) II < 

11*^771,nil ^m,n '-^m,n 

for all (in, n) G A, where Am,n are the evolutions operators of equation CH) derived 
from equation CB- Assume that 


X := max 


(m,n)GA I ^m,‘ 


k—n 


< 1 . 


Then 


for every (m,n) G A. 

Proof. Given n G No and a G \ {0}, let Qn,a be the space of functions 

X-. [n + r,+oo[z-t F^ 

such that 


X Yi - CX 


: mfi n> < +oo. 


\x\q^ _ = sup 


It is clear that Qn,a is a complete metric space with the metric defined by 


d(x,y) = |a; - _ = sup 

For every x G we define 


\Xm - Vn 


: m ^ n> . 


(J x'jrn — 


if TO = 


if TO > n. 


tiN) 








10 


ANTONIO J. G. BENTO, JOSE J. OLIVEIRA, AND CESAR M. SILVA 


where, for each i = 1,... ,N, 

m — 1 

h—71. 

Since for m > n we have 


I {Jx)m II ^ . max 


m — 1 

ii>i^>.ii + Eii<fe+iiiiir/l*^(^'=)ii 

^ k—n 

< ,J]iax^ I aW„||ai|| + E am^fe+i Lip(/i*^)||xfe|| | 


^ max < o' „ 
i=i,...,Ar ’ 




k—n 

m—1 


m—1 

ll«ill + E “m!fe+iLip(/i*^)a'fc_„||a|| \ x \ q ^ 

k—n 


k—n 

\\a 


^ I^ICn,—^ " 

and this implies that Jx belongs to Gn,a and 

^ 1 + a |a;|g 

Hence J: Gn,a Cn,Q- 

Now we prove that J is a contraction. For every x,y G Cn,a and every m > n 


( 20 ) 


we have 


< 


II ' 

- iJy)m\\ 

max < 

2=1,...,iV 1 

f m—1 

E ll<fc+illlir/i' 

1,, k—n 

max < 

2=1,...,iV 1 

f m — 1 

E Lip(/fe*’ 

1,, k—n 

max < 

2=1,...,iV 

f m — 1 

E “Sfc+i Lip(/i*’ 

1,, k—n 

am,nl|a||A|a^-2/le„^ 


-Vkl 


and thus 

\Jx- Jy\Q^_ < X\x-y\fo^_. 

Since A < 1, J is a contraction and by the Banach fixed point theorem J has a 
unique fixed point x*. By (EH)) it follows that the fixed point x* verifies 

|rie„.^< 

and this proves the theorem. □ 
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